Image capturing system and electronic endoscope system

ABSTRACT

An endoscope system includes an endoscope with an image pickup that generates image data by capturing an image of tissue illuminated with light containing wavelength regions spaced from and not overlapping with each other. The image pickup has an RGB filter, the wavelength regions include first and second regions corresponding to a B region and to a G region of the filter. A processor calculates an index representing a molar concentration ratio of first and second biological substances contained in the tissue based on the image data including information on the first and second regions. In the first region, image data B of the tissue varies depending on the molar concentration ratio, the second region contains isosbestic points of the tissue, and, in the second region, image data G has a constant value. The processor calculates the index based on the data B and G.

CROSS REFERENCE TO RELATED APPLICATIONS

This is a continuation application of U.S. patent application Ser. No.14/837,300, filed Aug. 27, 2015, which claims the benefit of JapanesePatent Application No. 2014-179036, filed Sep. 3, 2014. The entiredisclosure of each of the above-identified applications, including thespecification, drawings, and claims, is incorporated herein by referencein its entirety.

BACKGROUND OF THE INVENTION

The present invention relates to an image capturing system and anelectronic endoscope system.

Recently, an endoscope device (a spectroscopic endoscope device)equipped with a function to capture spectral images has been proposed.With such an spectroscopic endoscope device, it may be possible toobtain image information containing spectral property (e.g. areflectivity spectrum) of a living tissue such as a mucous membrane in adigestive organ. It is known that the reflectivity spectrum of a livingtissue reflects information concerning types or densities of componentscontained in the vicinity of a surface layer of the subject livingtissue targeted for measurement. Specifically, it is known thatabsorbance calculated from the reflectivity spectrum of the livingtissue can be obtained by linearly superimposing absorbance of aplurality of substances which constitute the living tissue.

It is known that a living tissue in a diseased portion is different froma living tissue in a healthy portion in regard to a composition andcomponent amounts. It is reported in prior research that abnormality ofa diseased portion, represented by a cancer, is deeply related to astate of blood, namely, a state of whole blood volume and oxygensaturation. It is a frequently used manner in spectroscopic analyticalchemistry to qualitatively and quantitatively analyze interesting twoliving tissues using spectroscopically characteristic amounts of the twoliving tissues in the visible light region. Therefore, it is possible toestimate whether or not a living tissue contains a diseased portion bycomparing a spectral property of blood of a living tissue containing adiseased portion with a spectral property of blood of a living tissuecontaining only a healthy portion.

The spectral images are constituted by a plurality of image informationcaptured with light having different wavelengths. As the wavelengthinformation contained in the spectral image (the numbers of wavelengthsused to obtained image information) increases, obtained informationconcerning the living tissue becomes more accurate. Japanese PatentProvisional Publication No. 2012-245223A (hereafter, referred to aspatent document 1) discloses an example of a spectroscopic endoscopedevice which obtains spectral images at intervals of wavelength of 5 nmwithin a wavelength region of 400 nm to 800 nm.

SUMMARY OF THE INVENTION

However, in order to obtain spectral images with a high degree ofwavelength resolution as described in patent document 1, it is necessaryto capture a number of images while changing wavelength of illuminationlight. Furthermore, since the required calculation amount for analyzingimages is larger, a long calculation time is needed for the analysis.That is, in order to obtain effective information for assistingdiagnosis, it is necessary to repeatedly execute relatively complicatedcapturing and calculation, which requires a long time.

The present invention is advantageous in that it provides an imagecapturing system and an electronic endoscope system capable ofobtaining, in a short time, image information representing distributionof biological substances, such as, distribution of oxygen saturation.

According to an aspect of the invention, there is provided an imagecapturing system, comprising: a light source device that emitsillumination light containing a plurality of wavelength regionsseparated from each other; an image pickup device that generates imagedata by capturing an image of a living tissue being a subjectilluminated with the illumination light, the image pickup device havingan RGB filter; and an image processing unit configured to calculate afirst index representing a molar concentration ratio of a firstbiological substance and a second biological substance contained in theliving tissue based on the image data generated by the image pickupdevice. In this configuration. The plurality of wavelength regionscomprise: a first wavelength region corresponding to a B filter of theRGB filter; and a second wavelength region corresponding to a G filterof the RGB filter. In the first wavelength region, a value of image dataB of the living tissue captured by a light-receiving element of theimage pickup device to which the B filter is attached varies dependingon the molar concentration ratio. The second wavelength region containsa plurality of isosbestic points of the living tissue, and, in thesecond wavelength region, a value of image data G of the living tissuecaptured by a light-receiving element of the image pickup device towhich the G filter is attached takes a constant value without dependingon the molar concentration ratio. The image processing unit isconfigured to calculate the first index having correlation with themolar concentration ratio based on the image data B and the image dataG.

With this configuration, it becomes possible to obtain, in a short time,image information representing distribution of biological substances,such as, distribution of oxygen saturation.

In at east one aspect, the light source device may comprises: a whitelight source; and an optical filter that separates the illuminationlight from white light emitted by the white light source.

In at least one aspect, the first index may be defined as a valueobtained by dividing the image data B by the image data G.

In at least one aspect, the plurality of wavelength regions may comprisea third wavelength region corresponding to an R filter of the RGBfilter. In this case, absorbance of the living tissue in the thirdwavelength region is regarded as almost zero, and the image processingunit is configured to calculate a second index having correlation with asum of molar concentrations of the first biological tissue and thesecond biological tissue based on the image data G and image data R ofthe living tissue captured by a light-receiving element of the imagepickup device to which the R filter is attached.

In at least one aspect, the second index may be defined as a valueobtained by dividing the image data G by the image data R.

In at least one aspect, the image capturing system may further comprisea memory storing base line image data BL_(R), BL_(G) and BL_(B)respectively corresponding to image data of R, G and B colors obtainedby capturing a color reference plate illuminated with the illuminationlight.

In at least one aspect, the image processing unit may be configured tocalculate he first index and the second index using, in place of theimage data R, the image data G and the image data B, standardizationimage data R_(S), G_(S) and B_(S) defined by following expressions:

R _(S) =R/BL_(R)

G _(S) =B/BL_(G)

B _(S) =B/BL_(B)

In at least one aspect, the image processing unit may be configured togenerate first index image data representing distribution of the firstindex in the living tissue.

In at least one aspect, the first index image data may be image datahaving a pixel value being the first index.

In at least one aspect, the first biological substance may beoxyhemoglobin, the second biological substance may be deoxyhemoglobin,and the first index may have correlation with oxygen saturation.

In at least one aspect, the image processing unit may be configured tocalculate a third index representing a degree of possibility of amalignant tumor of the living tissue based on the first index and thesecond index.

In at least one aspect, for a pixel having the first index lower than afirst reference value and the second index higher than a secondreference value, a value representing a high possibility of a malignanttumor may be assigned to the third index.

In at least one aspect, the second wavelength region may be comparted bya first pair of isosbestic points of the living tissue, and may includea second pair of isosbestic points of the living tissue lying within arange defined by the first pair of isosbestic points.

According to another aspect of the invention, there is provided anelectronic endoscope system, comprising: the above described imagecapturing system; and an electronic scope provided with the image pickupdevice.

BRIEF DESCRIPTION OF THE ACCOMPANYING DRAWINGS

FIG. 1 illustrates a transmittance spectrum of hemoglobin.

FIG. 2 is a graph plotting relationship between a transmitted lightamount of blood in a wavelength region W2 and oxygen saturation.

FIG. 3 is a graph plotting relationship between a transmitted lightamount of blood and oxygen saturation within a wavelength region W7.

FIG. 4 is a block diagram illustrating a configuration of an electronicendoscope system according to an embodiment of the invention.

FIG. 5 illustrates a transmittance spectrum of a color filter of a solidstate image pickup device.

FIG. 6 illustrates a transmittance spectrum of an optical filter.

FIG. 7 is a flowchart illustrating an analysis process according to theembodiment.

FIGS. 8A and 8B are examples of images generated by the electronicendoscope system according to the embodiment, in which FIG. 8A is anendoscopic image and FIG. 8B is an image of oxygen saturationdistribution.

FIG. 9 is a graph plotting results of simulation in which errors of anindex X caused when a transmittance wavelength region of the opticalfilter corresponding to the wavelength region W2 is shifted aresimulated.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, an embodiment according to the invention is described withreference to the accompanying drawings. In the following, an electronicendoscope system is explained as an embodiment of the invention by wayof example.

The electronic endoscope system according to the embodiment explainedbelow quantitatively analyzes biological information (e.g., oxygensaturation and blood volume) of a subject based on a plurality of images(three primary color images of R, G, B constituting one color image inthis embodiment) captured using light having different wavelengthregions, and the electronic endoscope system images and displaysanalysis results. In the quantitative analysis for, for example, oxygensaturation, explained below using the electronic endoscope systemaccording to the embodiment, a characteristic where a spectral propertyof blood in a visible light region continuously changes depending onoxygen saturation is utilized.

Calculation Principle of Spectral Property of Hemoglobin and OxygenSaturation Before explaining in detail a configuration of the electronicendoscope system according to the embodiment of the invention, acalculation principle of the spectral property of hemoglobin in avisible light region and oxygen saturation according to the embodimentis explained.

FIG. 1 illustrates a transmittance spectrum of hemoglobin. The spectralproperty of hemoglobin changes depending on oxygen saturation (a rationof oxyhemoglobin with respect to total hemoglobin). A waveform indicatedby a solid line in FIG. 1 represents a transmittance spectrum in thecase of oxygen saturation of 100% (i.e., oxyhemoglobin HbO₂), and awaveform indicated by a long dashed line represents a transmittancespectrum in the case of oxygen saturation of 0% (i.e., deoxyhemoglobinHb). Further, a waveform indicated by a short dashed line represents atransmittance spectrum of hemoglobin (a mixture of oxyhemoglobin anddeoxyhemoglobin) at an intermediate oxygen saturation (10%, 20%, 30%, .. . 90%).

The absorbance A of hemoglobin is calculated from light transmittance Tby the following equation (1):

A=−log T   (1)

Since the transmittance spectrum of hemoglobin shown in FIG. 1 is anoptical spectrum of two components where the total density of respectivecomponents (oxyhemoglobin and deoxyhemoglobin) takes a constant value,isosbestic points E1 (424 nm), E2 (452 nm), E3 (502 nm), E4 (528 nm), E5(546 nm), E6 (570 nm) and E7 (584 nm) appear. At each of the isosbesticpoints, the absorbance A (i.e., the transmittance T) takes a constantvalue without depending on a density ratio (i.e., oxygen saturation) ofrespective components. In this embodiment, a wavelength region from theisosbestic point E1 to the isosbestic point E2 is defined as awavelength region W1, a wavelength region from the isosbestic point E2to the isosbestic point E3 is defined as a wavelength region W2, awavelength region from the isosbestic point E3 to the isosbestic pointE4 is defined as a wavelength region W3, a wavelength region from theisosbestic point E4 to the isosbestic point E5 is defined as awavelength region W4, a wavelength region from the isosbestic point E5to the isosbestic point E6 is defined as a wavelength region W5, and awavelength region from the isosbestic point E6 to the isosbestic pointE7 is defined as a wavelength region W6.

Between the neighboring isosbestic points, the absorbance monotonouslyincreases or decreases in accordance with increase of oxygen saturation.Furthermore, between the neighboring isosbestic points, the absorbance Aof hemoglobin changes approximately linearly with respect to oxygensaturation. FIG. 2 is a graph where relationship between the transmittedlight amount of blood (the vertical axis) in the wavelength region W2and oxygen saturation (the horizontal axis) is plotted. The transmittedlight amount of the vertical axis represents an integrated value withinthe entire wavelength region W2. From the graph of FIG. 2, it isunderstood that the absorbance of hemoglobin decreases approximatelylinearly with respect to oxygen saturation in the wavelength region W2.It should be noted that, within the neighboring wavelength region W1,the absorbance of hemoglobin increases linearly with respect to oxygensaturation. Although, to be precise, light transmittance is a changingamount which conforms to Beer-Lambert Law, the light transmittance canbe regarded as changing approximately linearly within a comparativelynarrow wavelength region, such as within a range of 20 nm to 80 nm.

Focusing on the wavelength region from the isosbestic point E4 to theisosbestic point E7 (i.e., a continuous wavelength region from thewavelength region W4 to W6, which is defined as a wavelength region W7in this embodiment), the absorbance of blood increases monotonously inaccordance with increase of oxygen saturation within the wavelengthregions W4 and W6; however, the absorbance of blood inversely decreasesmonotonously within the wavelength region W5 in accordance with increaseof oxygen saturation. However, the inventor of the present invention hasfound that the decreasing amount of absorbance of blood in thewavelength region W5 is approximately equal to the increasing amount ofabsorbance of blood in the wavelength regions W4 and W6, and therefore,within the whole wavelength region W7, the absorbance of blood becomesapproximately constant value without depending on oxygen saturation.

FIG. 3 is a graph in which the relationship between the transmittedlight amount of blood (the vertical axis) and oxygen saturation (thehorizontal axis) within the wavelength region W7 is plotted. Thetransmitted light amount of the vertical axis is an integrated value ofthe whole wavelength region W7. The average value of the transmittedlight amount is 0.267 (an arbitrary unit), and the standard deviation is1.86×10⁻⁵. From FIG. 3, it is understood that, as a whole, thetransmitted light amount of blood becomes approximately constant in thewavelength region W7 without depending on oxygen saturation.

Furthermore, as shown in FIG. 1, in a wavelength region larger than orequal to approximately 630 nm (in particular in a region larger than orequal to 650 nm), the absorbance of hemoglobin is small, and almost nochange occurs in the light transmittance even when oxygen saturationchanges. When a xenon lamp is used for a white light source, asufficiently large amount of light can be obtained for a white lightsource within a wavelength region smaller than or equal to 750 nm (inparticular within a wavelength region smaller than or equal to 720 nm).Therefore, for example, a wavelength region of 650 nm to 720 nm can beused as a reference wavelength region for the transmitted light amountwhile regarding this wavelength region as a transparent region nothaving absorption for hemoglobin. In this embodiment, the wavelengthregion of 650 mn to 720 nm is defined as a wavelength region WR.

As described above, it is known that the absorbance A_(W2) of hemoglobinin the wavelength region W2 decreases linearly depending on increase ofoxygen saturation. Since the absorbance A_(W7) of hemoglobin in thewavelength region W7 (the wavelength regions W4 to W6) can be regardedas a constant value regardless of oxygen saturation, a value ofabsorbance A_(W2) with respect to absorbance A_(W7) represents an indexreflecting oxygen saturation. Specifically, an index X defined by thefollowing equation (2) represents oxygen saturation.

X=A _(W2) −A _(W7)   (2)

Therefore, by obtaining e relationship between oxygen saturation and theindex X experimentally in advance or by calculation, oxygen saturationcan be estimated from the index X.

General Configuration of Electronic Endoscope System

FIG. 4 is a block diagram illustrating a configuration of the electronicendoscope system 1 according to the embodiment. As shown in FIG. 4, theelectronic endoscope system 1 includes an electronic scope 100, aprocessor 200 and a monitor 300.

The processor 200 includes a system controller 202, a timing controller204, an image processing circuit 220, a lamp 208 and an optical filerdevice 260. The system controller 202 executes various programs storedin a memory 212, and controls totally the entire electronic endoscopesystem 1. The system controller 202 is connected to an operation panel214. The system controller 202 changes operation of the electronicendoscope system 1 and parameters for operation of the electronicendoscope system 1. The timing controller 204 outputs clock pulses foradjusting timings of various types of operation to respective circuitsin the electronic endoscope system 1.

The lamp 208 emits illumination light L after being activated by a lamppower igniter 206. For example, the lamp 208 is a high luminance lamp,such as, a xenon lamp, a halogen lamp, a mercury lamp or a metal halidelamp, or an LED (Light Emitting Diode). The illumination light L islight having a spectrum expanding principally from the visible lightregion to the invisible infrared region (or is white light including atleast a visible light region).

Between the lamp 208 and a condenser lens 210, the optical filer device260 is disposed. The optical filter device 260 includes a filer drivingunit 264 and an optical filter 262 attached to the filer driving unit264. The filer driving unit 264 is configured to be able to move theoptical filter 262 between a position (indicated by a solid line) on anoptical path of the illumination light L and a position (indicated by adashed line) retracted from the optical path by causing the opticalfiler 262 to slide in a direction perpendicular to the optical path, itshould be noted that the configuration of the filter driving unit 264 isnot limited to the above described configuration, but the filter drivingunit 264 may be configured such that the optical filter 262 is insertedinto or drawn from the optical path of the illumination light L byrotating the optical filer 262 about a rotation axis which is shiftedfrom the barycenter of the optical filter 262. The details about theoptical filter 262 are explained later.

The electronic endoscope system 1 according to the embodiment isconfigured to be able to operate in three operation modes: a normalobservation mode in which the white light emitted from the lamp 208 isused as the illumination light (normal light Ln) without change (orwhile removing an infrared component and/or a ultraviolet component) andendoscopic observation is conducted; a special observation mode in whichfiltered light Lf obtained by letting the white light pass the opticalfilter 262 is used as the illumination light and endoscopic observationis conducted; and a base line measurement mode in which a correctionvalue used for the special observation mode is obtained. The opticalfilter 262 is disposed at the retracted position from the optical pathin the normal observation mode, and is disposed on the optical path inthe special observation mode.

The illumination light L (the filtered light Lf or the normal light Ln)which has passed the optical filter 260 is converged onto an entranceend face of an LCB (Light Carrying Bundle) 102 by the condenser lens210, and is guided into the inside of the LCB 102.

The illumination light guided into the LCB 102 propagates through theLCB 102, and is emitted from an exit end face of the LCB 102 disposed ina tip portion of the electronic scope 100, and illuminates a subject viaa light distribution lens 104. Light returning from the subjectilluminated with the illumination light forms an optical image on alight-receiving surface of a solid state image pickup device 108 via anobjective lens 106.

The solid state image pickup device 108 is a single chip color CCD(Charge Coupled Device) image sensor having a Bayer type pixel array.The solid state image pickup device 108 accumulates charges according toa light amount of an optical image converted at each pixel on thelight-receiving surface, and generates and outputs an image signal(image data). The solid state image pickup device 108 has a so-calledon-chip filter in which an R filter for letting red light passtherethrough, a G filter for letting green light pass therethrough and aB filter for letting blue light pass therethrough are directly formed onrespective light receiving elements. The image signal generated by thesolid state image pickup device 108 includes an image signal R capturedby a light receiving element to which the R filter is attached, an imagesignal G captured by a light receiving element to which the G filter isattached and an image signal B captured by a light receiving element towhich the B filter is attached.

FIG. 5 illustrates transmittance spectrums of the R filter, the G filterand the B filter of the solid state image pickup device 108. The Rfilter is a filter for letting light having a wavelength region ofapproximately 600 nm or more including the wavelength region WR passtherethrough. The G filter is a filter for letting light having awavelength region of approximately 510 nm to 630 nm including thewavelength region W7 pass therethrough. The B filter is a filter forletting light having a wavelength region of approximately 510 nm or lessincluding the wavelength regions of W1 and W2 pass therethrough. Asdescribed later, the optical filter 262 has the optical property whereonly light in the three wavelength regions WR, W7 and W2 is selectivelyallowed to pass therethrough. An optical image of light having thewavelength regions of WR, W7 and W2 which has passed through the opticalfilter 262 is then picked up by the light receiving elements of thesolid state image pickup device 108 to which the R filter, the G filterand the B filter are attached, and is output as the image signals R, Gand B.

The solid state image pickup device 108 is not limited to the CCD imagesensor, but may be replaced with another type of image pickup devices,such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

As shown in FIG. 4, in a connection part of the electronic scope 100, adriver signal processing circuit 110 is provided. To the driver signalprocessing circuit 110, an image signal is input from the solid stateimage pickup device 108 in a field cycle. The driver signal processingcircuit 110 executes a predetermined process for the image signal inputfrom the solid state image pickup device 108, and outputs the processedimage signal to the image processing circuit 220 of the processor 200.

The driver signal processing circuit 110 accesses a memory 112 to readout unique information of the electronic scope 100. The uniqueinformation of the electronic scope 100 stored in the memory 112includes, for example, sensitivity and the pixel number of the solidstate image pickup device 108, operable field rates and a model number.The driver signal processing circuit 110 outputs the unique informationread from the memory 112 to the system controller 202.

The system controller 202 executes various types of operation based onthe unique information of the electronic scope 100, and generatescontrol signals. By using the generated control signals, the systemcontroller 202 controls the operation and timings of the variouscircuits in the processor 200 so that processes suitable for theelectronic scope connected to the processor 200 are executed.

The timing controller 204 supplies clock pulses to the driver signalprocessing circuit 110 in accordance with the timing control by thesystem controller In accordance with the clock pulses supplied from thetiming controller 204, the driver signal processing circuit 110 drivesand controls the solid state image pickup device 108 in synchronizationwith the field rate of video processed by the processor 200.

The image processing circuit 220 executes predetermined signalprocessing, such as color interpolation, a matrix operation and Y/Cseparation, and generates image data for monitor representation, andthen converts the generated image data for monitor representation to apredetermined video format signal. The converted video format signal isoutput to the monitor 300. As a result, an image of the subject isdisplayed on a display screen of the monitor 300.

The image processing circuit 220 includes an analysis processing circuit230. The analysis processing circuit 230 executes an analysis processThe analysis processing circuit 230 executes a spectroscopic analysisprocess based on the obtained image signals R (Red), G (Green) and B(Blue) in the special observation mode, calculates an index value havingcorrelation with oxygen saturation in a living tissue being the subject,and generates image data for visually displaying calculation results.

As described above, the electronic endoscope system 1 according to theembodiment is configured to be able to operate in the three operationmodes including the normal observation mode where the white light (thenormal light Ln) emitted from the lamp 208 is used as the illuminationlight, the special observation mode where spectroscopic analysis isexecuted using, as the illumination light, the filtered light Lfobtained by letting the white light pass the optical filter 262, and thebase line measurement mode for obtaining the correction value for thespecial observation. Switching between the modes is executed by a useroperation to an operation unit of the electronic scope 100 or theoperation panel 214 of the processor 200,

In the normal observation mode, the system controller 202 controls theoptical filter device 260 to retract the optical filter 262 from theoptical path, and performs image-capturing by illuminating the subjectwith the normal light Ln. Then, the system controller 202 converts theimage data obtained by the solid state image pickup device 108 to avideo signal after subjecting the image data to image processingaccording to need, and displays an image corresponding to the image dataon the monitor 300.

In the special observation mode and the base line measurement mode, thesystem controller 202 controls the optical filter device 260 to disposethe optical filter 262 on the optical path, and performs image-capturingby illuminating the subject with the filtered light Lf. Further, in thespecial observation mode, the analysis process which is described lateris executed based on the image data obtained by the solid state imagepickup device 108.

In the base line measurement mode, image-capturing is performed underillumination by the filtered light Lf using a color reference plate,such as, an achromatic diffusion plate or a standard reflector, and datato be used for a standardization process for the special observationmode which is described later is obtained.

The three primary color image data R(x, y), G(x, y) and B(x, y) obtainedby using the filtered light Lf in the base line measurement mode isstored respectively, as base line image data BL_(R)(x, y), BL_(G)(x, y)and BL_(B)(x, y), in an inside memory of the analysis processing circuit230. It should be noted that each of R(x, y), G(x, y) and B(x, y) is avalue of the image data at a pixel (x, y), and each of BL_(R)(x, y),BL_(G)(x, y) and BL_(B)(x, y) is a value of the base line image data ata pixel (x, y). The pixel (x, y) is identified by a coordinate x in thehorizontal direction and a coordinate y in the vertical direction.

Configuration and Property of Optical Filter

FIG. 6 illustrates a transmittance spectrum of the optical filter 262.The optical filter 262 is a so-called multi-peak dielectric multilayerfilter having an optical property that only light of the threewavelength regions of W2, W7 and WR is selectively allowed to pass theoptical filter 262 at least in the visible light wavelength region. Theoptical filter 262 has a flat transmittance in each of the wavelengthregions W2, W7 and WR, and the transmittance in the wavelength region W7is set to be lower than those of the other wavelength regions W2 and WR.This is because, since a light-emission spectrum of the white lightsource used in this embodiment has a peak in the wavelength region W7,the light amounts in the wavelength regions W2, W7 and WR after passingthrough the optical filter 262 can be maintained at approximately thesame light amount in the wavelength regions W2, W7 and WR and thereby anoise level of the received light data can be kept constant. Thetransmittance property of the filter can be determined based on thespectral wavelength luminance property of an actually used light sourceand the sensitivity property of a light-receiving element. As theoptical filter 262, another type of filter (e.g., an absorption typeoptical filter or an etalon filter in which a dielectric multilayer isused as a reflection coating of wavelength-selectivity) may be used.

Analysis Process in Special Observation Mode

Next, the analysis process executed in the special observation mode isexplained.

FIG. 7 is a flowchart illustrating the analysis process. In the analysisprocess, first the subject is captured by the solid state image pickupdevice 108, and the analysis processing circuit 230 obtains the threeprimary color image data R(x, y), G(x, y) and B(x, y) generated by thesolid state image pickup device 108 (S1).

Next, the analysis processing circuit 230 executes a pixel selectionprocess S2 in which pixels (x, y) targeted for the following analysisprocess (S3 to S6) are selected by using the image data R(x, y), G(x, y)and B(x, y).

Regarding a portion not containing blood or a portion in which color ofa tissue is dominantly affected by a substance other than hemoglobin, itis impossible to obtain a meaningful value even when oxygen saturationor blood volume is calculated from color of a pixel. Therefore,information from such a portion becomes noise. If such noise iscalculated and is presented to a medical doctor, the noise not onlybecomes an obstacle to appropriate diagnosis, but also causes a harmfulinfluence where a meaningless load is applied to the image processingcircuit 220 and thereby the processing speed is decreased. For thisreason, in an image generation process according to the embodiment,pixels (pixels in which the spectroscopic characteristic of hemoglobinis recorded) suitable for the analysis process are selected, and theanalysis process is executed only for the selected pixels.

In the pixel selection process S2, only pixels satisfying all of thefollowing conditions (3) to (5) are selected as target pixels for theanalysis process.

B(x, y)/G(x, y)>a ₁   (3)

R(x, y)/G(x, y)>a ₂   (4)

R(x, y)/B(x, y)>a ₃   (5)

where a1, a2 and a3 are positive constants.

The above described three conditions are set based on the magnitudecorrelation of “G component<B component<R component” in thetransmittance spectrum of blood. It should be noted that the pixelselection process S2 may be performed by using only one or two of theabove described three conditions. For example, only the condition (4)and/or the condition (5) may be used to execute the pixel selectionprocess S2 by focusing on red color unique to blood.

Next, the analysis processing circuit 230 executes a standardizationprocess S3. The standardization process S3 according to the embodimentis a process for making it possible to perform quantitative analysis bycorrecting the optical property e.g., transmittance of an optical filteror light-receiving sensitivity of an image pickup device) of theelectronic endoscope system 1 itself.

In the standardization process, the analysis processing circuit 230calculates standardization image data R_(S)(x, y) by the followingexpression (6) using the image data R(x, y) obtained by using thefiltered light Lf passed through the optical filter 262 and the baseline image data BL_(R)(x, y).

R _(S)(x, y)=R(x, y)/BL_(R)(x, y)   (6)

Similarly, from the following expressions (7) and (8), standardizationimage data G_(S)(x, y) and B_(S)(x, y) is calculated.

G _(S)(x, y)=G(x, y)/BL_(G)(x, y)   (7)

B _(S)(x, y)=B(x, y)/BL_(B)(x, y)   (8)

Although, in the following explanation, the standardization image dataR_(S)(x, y), G_(S)(x, y) and B_(S)(x, y) is used; however, calculationfor the index may be performed by using the image data R(x, y), G(x, y)and B(x, y) in place of the standardization data R_(S)(x, y), G_(S)(x,y) and B_(S)(x, y) without executing the standardization process.

Next, the analysis processing circuit 230 calculates a first index. Xhaving a correlation with oxygen saturation by the following expression(9) (S4).

X=B _(S)(x, y)/G _(S)(x, y)   (9)

The image data G(x, y) represents an optical image formed by light inthe wavelength region W7 which has passed through the optical filter262. The image data B(x, y) represents an optical image formed by lightin the wavelength region W2 which has passed through the optical filter262. As described above, the reflectivity of a living tissue withrespect to light in the wavelength region W2 (i.e., a value of the imagedata B) depends on both of oxygen saturation and a total hemoglobinamount. On the other hand, the reflectivity of a living tissue withrespect to light in the wavelength region W7 (i.e., a value of the imagedata G) does not depend on oxygen saturation, but depends on a totalhemoglobin amount. By dividing the standardization reflectivity B_(S) (avalue of the image data B after correction) by the standardizationreflectivity G_(S) (a value of the image data G after correction), itbecomes possible to cancel out contribution by a total hemoglobinamount. Furthermore, through such division, contribution by an incidentangle of the illumination light IL (the filtered light Lf) with respectto a living tissue or a surface condition of a living tissue can also becancelled out, and thereby only contribution by oxygen saturation can beextracted. Therefore, the first index X becomes a suitable index forrepresenting oxygen saturation.

Next, the analysis processing circuit 230 calculates a second index Yhaving correlation with the blood volume in a living tissue (a totalhemoglobin amount) from the following expression (10) (S5),

Y=G _(S)(x, y)/R _(S)(x, y)   (10)

As described above, the standardization reflectivity G_(S) does notdepend on oxygen saturation but depends on a total hemoglobin amount. Onthe other hand, since the standardization reflectivity R_(S) (a value ofthe image data R after correction) is reflectivity of a living tissue inthe wavelength region W7 in which there is almost no absorption forhemoglobin, the standardization reflectivity R_(S) does not depend onoxygen saturation nor on a total hemoglobin amount. By dividing thestandardization reflectivity G_(S) by the standardization reflectivityR_(S), contribution to reflectivity of a living tissue by an incidentangle of the illumination light IL to a living tissue or a surfacecondition of a living tissue can be canceled out, and thereby onlycontribution to a total hemoglobin amount can be extracted. Therefore,the second index Y becomes a suitable index for a total hemoglobinamount.

Next, the analysis processing circuit 230 calculates a third index Zrepresenting a possibility of a malignant tumor based on the first indexX and the second index Y.

It is known that a tissue of a malignant tumor has a larger amount oftotal hemoglobin than a normal tissue by vascularization, and oxygensaturation in a tissue of a malignant tumor is lower than that of anormal tissue since metabolism of oxygen in a tissue of a malignanttumor is remarkable. For this reason, the analysis processing circuit230 extracts a pixel having the first index X representing oxygensaturation calculated by the expression (9) is smaller than apredetermined reference value (a first reference value) and the secondindex Y representing a total hemoglobin amount calculated by theexpression (10) larger than a predetermined reference value (a secondreference value), and assigns “1” representing a possibility of amalignant tumor to the third index Z of the extracted pixel and assigns“0” to the third indexes X of the other pixels.

Each of the first index X, the second index Y and the third index Z maybe defined as a binary index, and the third index Z may be calculated asa logical product or a logical sum of the first index X and the secondindex Y. In this case, for example, by defining X=1 (low oxygensaturation) when the right term of the expression (9) is smaller than apredetermined value, defining X=0 (a normal value) when the right termof the expression (9) is larger than or equal to the predeterminedvalue, defining Y=1 (large blood volume) when the right term of theexpression (10) is larger than or equal to a predetermined value anddefining Y=0 (a normal value) when the right term of the expression (10)is smaller than the predetermined value, Z can be calculated as Z=X·Y(logical product) or Z=X+Y (logical sum).

The foregoing is an example where the third index Z is defined as abinary index; however, the third index Z may be defined as a multiplevalue (or as a continuous value) index representing a degree ofpossibility of a malignant tumor. In this case, the third index Z(x, y)representing a degree of possibility of a malignant tumor may becalculated based on a deviation from the first reference value or anaverage of the first index X(x, y) and a deviation from the secondreference value or an average of the second index Y(x, y). For example,the third index Z(x, y) may be calculated as a sum or a product of thedeviation of the first index X(x, y) and the deviation of the secondindex Y(x, y).

Next, the analysis processing circuit 230 generates index image data inwhich one of the first index X(x, y), the second index Y (x, y) and thethird index Z(x, y) is designated in advance by a user as a pixel value(luminance) (S7). It should be noted that in step S7 index image datafor all the first index X(x, y), the second index Y (x, y) and the thirdindex Z(x, y) may be generated.

Next, the analysis processing circuit 230 executes a color correctionprocess S8 for the image data R(x, y), G(x, y) and B(x, y). Since thefiltered light Lf which has passed through the optical filter 262includes the three primary color spectrum components of R (thewavelength region WR), G (the wavelength region W7) and B (thewavelength region W2), it is possible to obtain a color image using thefiltered light Lf. However, since the band of the spectrum of thefiltered light Lf is restricted, an image captured by using the filteredlight Lf has a relatively unnatural tone of color in comparison with animage captured by using the normal light Ln. For this reason, in thisembodiment, the color correction process S8 is executed for the imagedata R(x, y), G(x, y) and B(x, y) obtained by using the filtered lightLf so that a tone of color of an image captured by using the filteredlight Lf approaches a tone of color of an image captured by using thenormal light Ln.

The color correction process S8 is executed, for example, by addingcorrection values C_(R), C_(G) and C_(B) obtained in advance to theimage data R(x, y), G(x, y) and B(x, y) or multiplying the image dataR(x, y), G(x, y) and B(x, y) by the correction values C_(R), C_(G) andC_(B). Alternatively, by preparing a color matrix Mf for the filteredlight Lf in addition to a color matrix Mn for the normal light Ln, thecolor collection may be performed through a color matrix operation. Thecorrection values C_(R), C_(G) and C_(B) and the color matrix Mf are setin advance based on image data which the electronic endoscope system Iobtains by capturing a color reference plate illuminated with thefiltered light Lf, and are stored in the inside memory of the analysisprocessing circuit 230. The analysis process may be configured to omitthe color correction process S8.

Next, the analysis processing circuit 230 generates image data fordisplaying on the monitor 300 based on the image data subjected to thecolor correction process S8 and the index image data generated in stepS7 and the like (S9). In an image data generation process S9, varioustypes of image data for, for example, multiple image representation inwhich an endoscopic image after color correction and one or more typesof index images are displayed side by side on one screen, endoscopicimage representation in which only an endoscopic image after colorcorrection is displayed, and index image representation in which onlyone or more types of index images designated by the user is displayed,may be generated. The type of image data to be generated is selected bya user operation to the operation unit of the electronic scope 100 or tothe operation panel 214 of the processor 200. Furthermore, relatedinformation, such as, patient information, input from, for example, theoperation panel 214 of the processor 200 is displayed on the displayscreen in a superimposing manner.

FIGS. 8A and 8B are examples displayed on the monitor 300. Specifically,FIG. 8A is an endoscopic image, and FIG. 8B is an index image of thefirst index X(x, y). Each of FIGS. 8A and 8B is obtained by observingthe right hand in a state where a portion around a proximalinterphalangeal joint of the middle finger is pressed by a rubber band.FIG. 8B shows a state where oxygen saturation becomes lower in a distalpart with respect to the pressed part of the right middle finger due tohampering of blood flow by pressure. Furthermore, it can be read that,on a proximal side immediately near the pressed part, arterial bloodstays and therefore Oxygen saturation becomes high locally.

By conducting endoscopic observation while displaying two screensincluding an endoscopic image and an index image on the monitor 300 inthe special observation mode, it becomes possible to securely find amalignant tumor showing characteristic change in a total hemoglobinamount and oxygen saturation. Furthermore, when a portion having apossibility of a malignant tumor is found, by switching from the specialobservation mode to the normal observation mode through an operation tothe electronic scope 100 and thereby displaying a normal observationimage having proper color reproducibility on the entire screen, moredelicate diagnosis can be conducted. The electronic endoscope system 1according to the embodiment is configured to be able to switch betweenthe normal observation mode and the special observation mode easily andrapidly by simply changing an image processing manner while inserting orretracting the optical filter 262 to or from the optical path by anoperation to the electronic scope 100.

Furthermore, in the electronic endoscope system 1, the multi-peakoptical filter 262 which divides light into the three wavelength regionsW2, W7 and WR is employed, and further the configuration where light ofthe three wavelength regions W2, W7 and WR respectively passes the Bfilter, the G filter and the R filter of the solid state image pickupdevice 108 is employed. With this configuration, it becomes possible togenerate a frame of endoscopic image and an index image by capturing oneframe (2 fields). Accordingly, a video image having a high effectiveframe rate can be obtained.

The three transmittance wavelength regions of the optical filter 262 maybe shifted somewhat from the wavelength regions W2, W7 and WR defined bythe isosbestic points of hemoglobin as long as the indexes X and Yhaving a desired degree of accuracy can be obtained. FIG. 9 is a graphplotting results of simulation in which errors of the index X causedwhen the transmittance wavelength region of the optical filter 262corresponding to the wavelength region W2 (452 nm to 502 nm) is shiftedare simulated. In FIG. 9, the horizontal axis represents oxygensaturation, and the vertical axis represents an error of the index X.

In FIG. 9, a plotted point A represents an error caused when the opticalfilter 262 configured to let light of a wavelength region of 454 nm to504 nm shifted to the longer wavelength side by 2 nm from the wavelengthregion W2 pass therethrough is used. A plotted point B represents anerror caused when the optical filter 262 configured to let light of awavelength region of 446 nm to 496 nm shifted to the shorter wavelengthside by 6 nm from the wavelength region. W2 pass therethrough is used. Aplotted point C represents an error caused when the optical filter 262configured to let light of a wavelength region of 440 nm to 490 nmshifted to the shorter wavelength side by 12 nm from the wavelengthregion W2 pass therethrough is used. The error of each of the plottedpoints is smaller than 8%. Furthermore, it is understood that the graphshows a tendency that the error becomes a little larger in the casewhere the transmittance wavelength region is shifted to the longerwavelength side relative to the case where the transmittance wavelengthregion is shifted to the shorter wavelength side.

It is reported that oxygen saturation in a normal living tissue islarger than or equal to approximately 90%, and, by contrast, oxygensaturation in a malignant tumor is smaller than or equal toapproximately 40%. Therefore, it is considered that, if the error of theindex X is suppressed to be smaller than or equal to approximately ±5%,the index becomes sufficiently usable in practical use for the purposeof identifying a malignant tumor.

Therefore, regarding the shift amount of the transmittance wavelengthregion of the optical filter 262 corresponding to the wavelength regionW2, a range of ±12 nm (or a range of −12 nm to +10 nm, considering thatthe error becomes larger when the transmittance wavelength region shiftsto the longer wavelength side) is sufficiently acceptable. At least in arange of the plotted condition in FIG. 9 (i.e., in the range of thewavelength shift amount of −12 nm to +2 nm), the error of the index Xcan be suppressed to a sufficiently usable range in practical use.

Furthermore, although FIG. 9 shows simulation results when thewavelength is shifted without changing the wavelength width; however, itis considered that substantially the same tolerance can be applied toshift of the wavelength region accompanying increase or decrease of awavelength width. That is, it is considered that an error within a rangeof ±12 nm (preferably, −12 nm to +10 nm, and more preferably, −12 nm to+2 nm) is allowed for each of a longer wavelength edge and a shorterwavelength edge of a transmittance wavelength region corresponding tothe wavelength region W2.

Furthermore, it is considered that, regarding the wavelength regioncorresponding to each of the wavelength region W7 and the wavelengthregion WR, at least substantially the same tolerance can be accepted.

According to the above described embodiment, a relatively widewavelength region having a property that absorption of the entirewavelength region does not depend on oxygen saturation is used in placeof an isosbestic point of which wavelength region is extremely narrow.Therefore, it becomes unnecessary to use light having an extremelynarrow band and having a high spectral energy density, such as laser,and it becomes possible to accurately estimate information concerninghemoglobin density (blood density) without being affected by oxygensaturation even when narrowband light having a low spectral energydensity, such as, light whose wavelength region is separated from whitelight by a bandpass filter, is used.

The foregoing is the explanation about the embodiments of the invention.The invention is not limited to the above described embodiments, but canbe varied in various ways within the scope of the invention. Forexample, the invention includes a combination of embodiments explicitlydescribed in this specification and embodiments easily realized from theabove described embodiment.

In the above described embodiment, the example where the wavelengthregion W2 is used as a wavelength region for blue color used for thespecial observation mode is described; however, the wavelength region W1may be used in place of the wavelength region W2. The wavelength regionW1 has a property that a difference between absorption of oxyhemoglobinand absorption of deoxyhemoglobin is larger than that of the wavelengthregion W2. Therefore, by using the wavelength region W1, it becomespossible to detect change in oxygen saturation with higher sensitivity.On the other hand, the wavelength region W2 has a property thatabsorption of hemoglobin is smaller than that of the wavelength region.W1, and, when a xenon lamp is used as the lamp 208, the light emissionintensity of the lamp 208 becomes higher and therefore a larger lightamount can be obtained. For this reason, by using the wavelength regionW2, it becomes possible to detect oxygen saturation with a high degreeof accuracy (with low noise).

The above described embodiment is an example where a spectroscopicanalysis result is displayed as an gray scale index image or amonochrome index image; however, the displaying manner of analysisresult is not limited to such an example. For example, the image dataR(x, y), G(x, y) and B(x, y) may be altered depending on the indexvalue. For example, for a pixel whose index value exceeds a referencevalue, a process for increasing the brightness of the pixel, a processfor changing color phase of the pixel (e.g., a process for intensifyingredness by increasing a red component or a process for rotating colorphase by a predetermined angle), or a process for blinking the pixel (ora process for periodically changing color phase) may be performed.

In the above described embodiment, the solid state image pickup device108 is explained as a solid state image pickup device for capturing acolor image having an on-chip color filter; however, the presentinvention is not limited to such a configuration. For example, a solidstate image pickup device for capturing a monochrome image having aso-called frame sequential type color filter may be used. The colorfilter may be disposed at any point on the optical path extending fromthe lamp 208 to the solid state image pickup device 108.

In the above described embodiment, the optical filter device 260 isdisposed on the light source side to filter the illumination light IL;however, the present invention is not limited to such a configuration.The optical filter 260 may be disposed on an image pickup device side,and filtering may be performed for returning right from the subject.

In the above described embodiment, the present invention is applied toan electronic endoscope system which is an example of a digital camera;however, the present invention may be applied to a system in whichanother type of digital camera (e.g., a digital single-lens reflexcamera or a digital video camera) is used. For example, when the presentinvention is applied to a digital still camera, observation for a bodysurface tissue and observation (e.g., rapid examination for brain bloodvolume) for a brain tissue during craniotomy can be conducted.

This application claims priority of Japanese Patent Application No.P2014-179036, filed on Sep. 3, 2014. The entire subject matter of theapplication is incorporated herein by reference.

What is claimed:
 1. An electronic endoscope system, comprising: anelectronic endoscope provided with an image pickup device, the imagepickup device generating image data by capturing an image of a livingtissue as a subject illuminated with illumination light, which containsa plurality of wavelength regions spaced from each other and notoverlapping with each other, the image pickup device having an RGBfilter, the plurality of wavelength regions comprising a firstwavelength region corresponding to a B filter of the RGB filter, and asecond wavelength region corresponding to a G filter of the RGB filter;and an image processor configured to calculate a first indexrepresenting a molar concentration ratio of a first biological substanceand a second biological substance contained in the living tissue basedon the image data which includes information on the first wavelengthregion and the second wavelength region obtained by the image pickupdevice in a single frame, wherein: in the first wavelength region, avalue of image data B of the living tissue captured by a light-receivingelement of the image pickup device to which the B filter is attachedvaries depending on the molar concentration ratio; the second wavelengthregion contains a plurality of isosbestic points of the living tissue,and, in the second wavelength region, a value of image data G of theliving tissue captured by a light-receiving element of the image pickupdevice to which the G filter is attached has a constant valueindependent of the molar concentration ratio; and the image processor isconfigured to calculate the first index representing the molarconcentration ratio based on the image data B and the image data G. 2.The electronic endoscope system according to claim 1, wherein the firstindex is defined as a value obtained by dividing the image data B by theimage data G.
 3. The electronic endoscope system according to claim 1,wherein: the plurality of wavelength regions comprise a third wavelengthregion corresponding to an R filter of the RGB filter; absorbance of theliving tissue in the third wavelength region is substantially zero; andthe image processor is configured to calculate a second index havingcorrelation with a sum of molar concentrations of the first biologicaltissue and the second biological tissue based on the image data G andimage data R of the living tissue captured by a light-receiving elementof the image pickup device to which the R filter is attached.
 4. Theelectronic endoscope system according to claim 3, wherein the secondindex is defined as a value obtained by dividing the image data G by theimage data R.
 5. The electronic endoscope system according to claim 3,further comprising a memory that stores base line image data BLR, BLGand BLB respectively corresponding to image data of R, G and B colorsobtained by capturing a color reference plate illuminated with theillumination light.
 6. The electronic endoscope system according toclaim 5, wherein the image processor is configured to calculate thefirst index and the second index by using, as the image data R, theimage data G and the image data B, standardization image data R_(S),G_(S) and B_(S) defined by following expressions:R _(S) =R/BL_(R)G _(S) =G/BL_(G)B _(S) =B/BL_(B)
 7. The electronic endoscope system according to claim1, wherein the image processor is configured to generate first indeximage data representing distribution of the first index in the livingtissue.
 8. The electronic endoscope system according to claim 7, whereinthe first index image data is image data having a pixel value being thefirst index.
 9. The electronic endoscope system according to claim 1,wherein: the first biological substance is oxyhemoglobin; the secondbiological substance is deoxyhemoglobin; and the first index hascorrelation with oxygen saturation.
 10. The electronic endoscope systemaccording to claim 3, wherein the image processor is configured tocalculate a third index representing a degree of possibility of amalignant tumor of the living tissue based on the first index and thesecond index.
 11. The electronic endoscope system according to claim 10,wherein, for a pixel having the first index lower than a first referencevalue and the second index higher than a second reference value, a valuerepresenting a high possibility of a malignant tumor is assigned to thethird index.
 12. The electronic endoscope system according to claim 1,wherein the second wavelength region is comparted by a first pair ofisosbestic points of the living tissue, and includes a second pair ofisosbestic points of the living tissue lying within a range defined bythe first pair of isosbestic points.
 13. The electronic endoscope systemaccording to claim 1, wherein the plurality of wavelength regionscomprise a third wavelength region corresponding to an R filter of theRGB filter; absorbance of the living tissue in the third wavelengthregion is substantially zero; and the image processor is configured tocalculate a second index having correlation with a sum of molarconcentrations of the first biological tissue and the second biologicaltissue based on the image data G and the image data R of the livingtissue captured by a light receiving element of the image pickup deviceto which the R filter is attached.
 14. The electronic endoscope systemaccording to claim 13, wherein the second index is comprises a valueobtained by dividing the image data G by the image data R.
 15. Theelectronic endoscope system according to claim 13, further comprising amemory that stores baseline image data BL_(R), BL_(G), and BL_(B)respectively corresponding to the image data of R, G, and B colorsobtained by capturing a color reference plate illuminated with theillumination light.
 16. The electronic endoscope system according toclaim 13, wherein the image processor is configured to calculate a thirdindex representing a degree of possibility of a malignant tumor of theliving tissue based on the first index and the second index.
 17. Theelectronic endoscope system according to claim 13, wherein the firstbiological substance is oxyhemoglobin, the second biological substanceis deoxyhemoglobin and the first index has a correlation with oxygensaturation.